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ENVELOPES AND PRINTED DOCUMENTS EMPLOYING DIGITAL 
WATERMARKS, AND RELATED METHODS - 

Related Application Data 
5 In the U.S., this application is a continuation-in-part of copending applications 

09/567,405, filed May 8, 2000, and 09/629,649, filed August 1, 2000. 

Field of the Invention 
The present invention relates to use of digital watermark technology in 
1 0 conjunction with envelopes and other documents. 



Background and Summary of the Invention 
Computer printers have long heen used to print addresses on envelopes. With 
the advent of digital postage, use of printers with envelopes is increasing still further. 
15 Digital postage technology is available firom a nimiber of vendors including 

Pitney Bowes, E-Stamp, Stamps.com and Escher Laboratories (of Escher Group, Ltd.), 
and is detailed in various patent publications including 5,982,506, 5,825,893, 
5,819,240, 5,801,364, 5,774,886, 5,682,318, 5,978,781, and WO 99/18543AL 
Digital watermarking technology is used, in accordance with certain 
20 embodiments of tiie present invention, to uicrease the security of, and augment the 
functionahty associated with, computer printing of envelopes and postage. 

In accordance with one aspect of the invention, traceability of digital postage is 
enhanced by serialization, i.e., embedding a serial number code or other indicia that 
uniquely and covertly links the printed postage to some device or software in the users' 
25 possession, or that identifies the user. This device can be a printer, personal computer, 
or hardware security device used in printing the postage. In an exemplary embodiment, 
digital watermarking of the sort detailed m the cited patents and applications is used to 
embed the code at the time the postage is printed. The embedded data would only be 
detectable to investigators equipped with special readers for spot checking documents 
30 or investigating coimterfeits. 

In accordance with another aspect of the invention, security of digital postage 
against reproduction is enhanced through use of "fragile" digital watermarks. (A 
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"fragile" digital watermark is one designed, e.g., not to fully withstand the 
scanning/printing operations associated with photocopying or PC-based scanning and 
printing.) Such a watermark may be employed to provide forensic evidence that 
printed postage is not original. 
5 in accordance with yet another aspect of the invention, watermark technology is 

employed to prevent - outright - the photocopying or other duplication of digital 
postage. This result is achieved by encoding on envelopes a **do not copy" watermark 
to which photocopiers, scanners, printers, and other computer devices are alert. If such 
a watermark is encountered, the device will refuse to operate, or will otherwise 
1 0 interfere with the reproduction operation. 

In accordance with still another aspect of the invention, watermarking on an 
envelope is employed as an element of a franking mark (postal mark) - one that may 
. stay within or extend well beyond the comer location typically associated with such 
marks. 

15 In accordance with yet another aspect of the invention, watermarking on an 

envelope can serve as a portal to a corresponding intemet site or internet-based 
appUcation. That is, a printed document with an embedded watermark can be held up 
to a web cam, or scanned by a scaimer, and instantly link a user to Intemet sites or 
applications, hnportantly, information received in this manner is not subject to the 

20 delays associated with physical mail delivery, but can convey up-to-the-minute 
information. 

In accordance with still another aspect of the invention, an envelope watermark 
serves to convey an identifier that is used to access associated data in a database. In 
one particular appUcation, the index number identifies the recipient. Thus, for example, 

25 an envelope can be addressed simply by watermarking it with a imique recipient 

designator, e.g., JOHNQPtIBLIC843. Processing equipment in the postal system can 
read the watermark, query a database with the designator, and determine thereby the 
recipient's physical address. (If desired, the address thereby discemed can be printed 
on the envelope.) One advantage to this arrangement is that distribution of Change of 

30 Address cards would be a thing of the past. If a person moves across country, a single 
record id the database is changed. All mail to that recipient automatically is directed to 
the new physical address. 
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In accordance with yet another embodiment, digital watermarks on envelopes 
can be applied by specialized printers (e.g., postal metering devices), or by using 
common office printers (e.g., laser, ink-jet). In such systems, the watermark embedder 
software may be integrated into the printing device, or can be resident on an associated 
5 computer system. The software is desirably secured against tampering using various 
anti-hacking techniques. The production of the digital watermark may not be optional 
(i.e., it may be applied without user control), and the payload can be tailored in 
accordance with the amount of postage, device/soflware/user information, or other 
appHcation information. Such a system may also include an application that calibrates 

1 0 printing of the watermark to the user' s specific printer or software, thus 

accommodating a wide range of usage scenarios. A hardware security device used, 
e.g., to store postage value (e.g., a digital vault) may also be employed by the 
watermark embedding system, e.g., as a source of secure and potentially unique data 
used in encoding the digital watermark. 

15 In accordance with still further embodiments of the invention, the principles 

described herein are also applicable to other printed value documents (e.g., tickets and 
coupons), especially those that are printed on demand. Such documents may be printed 
at home, at special kiosks (e.g., in-store), or by commercial printing estabUshments 
(i.e., mass produced). Among other functionality, the watermarks in such value 

20 documents can be used by investigators to distinguish originals from reproductions (by 
use of fragile watermarks), to authenticate documents (e.g., in ticket reading machines), 
and to Hnk to associated intemet resources. The watermarks can also be linked to other 
information (e.g., event date, seat number, product code, etc.) textually printed on the 
document, or present on the document in some machine readable form (e.g., barcodes). 

25 In such case, the watermark can be used to detect document alteration by checking for 
discrepancy between the watermark-encoded information, and that otherwise conveyed 
by the document. 

In accordance with yet other embodiments of the invention, blank printing stock 
(e.g., other than envelopes) can be digitally watermarked to serve various ends. The 
30 watermarking continues to be detectable after the blank stock is thereafter printed. 
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The features just-described can be employed alone or in various combinations. 
The foregoing and additional features and advantages of the present invention 
will be more readily apparent from the following detailed description. 

5 Detaiiled Description 

Digital watermarking technology, a form of steganography, encompasses a 
great variety of techniques by which plural bits of digital data are hidden in some other 
object without leaving human-apparent evidence of alteration or data representation. 

Digital watermarking of envelopes can be effected in numerous ways, including 

10 by ink (clear, optically-opaque, ERAJV-opaque), by texturing, by laminate layers, etc. 
The watermarking can be formed as part of other markings on the onivelope (e.g., 
franking marks, graphics, text, etc.), or can be applied irrespective of such other 
markings. Watermarking can be effected at any time in an envelope's life, including at 
the time of media-making (e.g., paper making or Tyvek formation), at the time of 

1 5 envelope making, at the time of consumer use (either before, during, or after the 
appUcation of other envelope markings), and thereafter (e.g., in the course of postal 
service processing). 

The watermarking may span all of one side (or both sides) of an envelope, or 
may be localized, e.g., in the areas typically associated with postage, retum address, 

20 and recipient address. An envelope may convey a single watermark, or several may be 
used, e.g., conveying different information or serving different purposes in different 
regions. Several different watermarking technologies can be employed on a single 
envelope, e.g., the envelope's texture can convey one type of information, and tinting 
printed on the envelope can convey a second type of information. 

25 Any print- or physical media-watermark technology can be employed in 

conjunction with the present invention. Representative watermarking technologies 
suitable for such use are detailed in the assignee's patent 5,862,260, and in applications 
09/074,034 (counterpart published by PCT as W09953428), 09/127,502 
(WO0007356), 09/503,881 (PCT/USOl/00884), 09/562,516, and 09/562,524. A great 

30 many other watermarking technologies are familiar to those skilled in the digital 
watermarking art. 
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In accordance with one aspect of the invention, traceabihty of digital postage is 
enhanced by serialization, i.e., embedding a serial number code or other indicia that 
uniquely and covertly links the printed postage to some device in the users' possession. 
In one such embodiment, the watermark serves to convey an identifier of a 
5 printer, personal computer, postage vault, or other device used in printing postage. The 
identifier can be a registration number, a serial number, an account number, etc. The 
watermark can also serve to convey an identifier associated with particular software 
employed by the user. And/or, the watermark can also serve to identify the user. Other 
forensic information can likewise be encoded. 
10 The encoded information can directly correspond to the device, etc., or the 

relationship can be established through a remote database (e.g., the identifier can be an 
index number that, when looked-up in a database, yields the registered owner name and 
address of a particular device). 

Typically, such a watermark is "private," i.e., it is readable only to selected 
15 classes of persons who have access to secret data, such as a private key. Postal 

investigators and the like would be able to read such data (e.g., by using a specialized 
reader system, or by using a conventional reader system equipped with the private 
information), but the general public would not. 

In other embodimCTits, the watermark is public, but general use thereof is 
20 limited because a database needed to interpret the encoded data is not publicly 
accessible. 

As indicated above, this forensic watermark can take various forms. For 
example, it can form part of the fi-anking indicia printed on the envelope, or can be 
separate firom such indicia. It can be limited to the franking comer of the envelope, or 

25 can be located in a different location, or span a larger area. One particular 

implementation deposits a light splattering of tiny ink droplets oy&v an area. These 
droplets are sufficient to form a computer-detectable pattem, but are not conspicuous 
(or preferably even visible) to human observers. In this, and other embodiments, the 
invisibility of the markings can be enhanced by using inks responsive to ultraviolet or 

30 infrared illumination, as more particularly detailed in cited appUcation 09/562,5 16. 

In most applications, the forensic watermark is appUed automatically as part of 
another envelope processing activity. Thus, for example, such fimctionality can be 
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provided in software used to print addresses on envelopes, or apply digital postage to 
envelopes. The software can be of the consumer variety (e.g., Microsoft Word), or it 
can be system or device instructions invoked as part of the printing operation (e.g., 
printer driver software, or firmware associated with a printer's microprocessor.) As the 
5 user-intended information is being printed, the forensic marking is also being apphed. 
Thereafter, if an issue arises as to the source of an envelope, or postal indicia thereon, 
the forensic information can be checked to aid in such investigation. 

In accordance with a second aspect of the invention, security of digital postage 
against reproduction is enhanced through use of "fragile" digital watermarks. 

10 As noted, a "fragile" digital watermark is one designed not to fully withstand 

the scanning/printing operations associated with photocopying. (The use of fragile 
watermarks is detailed in the assignee's applications 09/234,780 (W09936876), 
09/287,940, 09/433,104 (PCT/USOO/29454), 09/498,223, 09/625,577, 60/198,138, 
09/645,779, 09/689,226, 09/689,250 and 09/689,293. If markings (e.g., legitimate 

15 franking indicia) incorporating such a watermark are photocopied or otherwise 

reproduced from one envelope onto a second envelope, the copy will either not ftdly 
include the watermark, or tiie watermark wiU be changed in a way that indicates it is a 
copy. Processing equipment in the postal system can be alert to such copies (which are 
identified by the absence or modification of the fragile watermark), and cull them from 

20 the properly-franked mail. Likewise, fraud or counterfeit investigators can use special 
readers to verify originaUty and detect copies. 

A watermark may be made fragile in numerous ways. One form of fragility 
rehes on low watermark amphtude. That is, the strength of the watermark is only 
marginally above the noinimum needed for detection. If any significant fraction of the 

25 signal is lost, as typically occurs in photocopying operations, the watermark becomes 
unreadable. 

Another form of firagiUty rehes on the watermark's frequency spectrum. High 
frequencies are typically attenuated in the various sampUng operations associated with 
digital scanning and printing. Even a high amphtude watermark signal can be 
30 significantly impaired, and rendered unreadable, by such photocopying operations. 

The foregoing are but two of many different approaches. The above-cited 
applications disclose many others. The particular fragile watermark used can be 
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tailored in accordance with the type of scanning and printing anticipated in 
unauthorized reproduction. 

Likewise, the fragile watermark can be implemented in various ways. For 
example, the watermark can be implemented by varying thicknesses of lines, adding 
5 dots or speckles or ink, or modulating the brightness of printed pixels. (Such 
watermarking arrangements are further detailed in applications 09/074,034 and 
09/127,502.) Or the watermark can be formed by texturing of the substrate. Such 
texturing can be appUed in various ways. One is by a mechanism integrated with the 
printer, e.g., one that impresses the medium with a pinch roller or other pressure- 

10 applying means. Another is during fabrication of the paper, e.g., by texturing 

dewatering elements in the paper making machinery to impress a desired pattern on the 
medium. (One such arrangement is detailed in application 09/437,357, filed November 
10, 1999 (PCT/USOO/30694),) 

As indicated, processing equipment in the postal system (e.g., document sorters 

15 and postal processing machines) can routinely scan envelopes bearing digital postage 
for the presence of the expected fragile watermark. Any envelopes found to be missing 
the watermark can be culled for investigation. This analysis may include watermark- 
reading software that infers information about the type of reproduction employed by 
reference to the attributes of any remaining fragile watermark signal, 

20 In accordance with a third aspect of the invention, watermark technology is 

employed to prevent - outright - the photocopying or other duplication of digital 
postage. This result is achieved by encoding on envelopes a "do not copy" watermark 
to which photocopiers, scanners, printers, imaging software, or other computer devices 
are alert. If such a watermark is encountered, the device will refuse to operate, or will 

25 otherwise interfere with the reproduction operation. 

Such watermark-based "do not copy* systems are further detailed in 
applications 09/074,034, 09/127,502, 09/185,380 (WO0026749) and 09/287,940. The 
detection of the watermark can occur in various, and preferably numerous, locations in 
likely reproduction systems. In a desktop computer system, for example, image data 

30 may be analyzed for such a watermark by software in the scanner (e.g., scanner driver 
software), software in the computer (e.g., TWAIN interface software, operating system 
software, image editing software, internet browser software, printer driver software). 
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and software in the printer (e.g., printer firmware). If any of these detectors encounters 
image data that has a "do not copy" watermark encoded therein, the detector will 
interfere with its reproduction (e.g., by discontinuing the process, by scarring the 
image, by hiding tracer data for later forensic use, etc.) 
5 The use of a watermark to indicate that an indicia should not be copied is 

desirable, but not necessary. Other hallmarks can be employed. For example, devices 
used in reproduction can be alert to the franking indicia itself and, if encountered, 
interfere with duplication. 

In accordance with a fourth aspect of the invention, security is enhanced by 

10 associating (cryptographically or otherwise) a digital watermark formed on envelope 
stock (e.g., by printing or texturing) with data conveyed in a postal franking mark (e.g., 
a 2D bar code). In such case, for example, the envelope can be authorized for use only 
in conjxmction with a certain printer, a certain postal meter, a certain postal account, a 
certain software, etc. If the envelope stock is diverted to another use (e.g., used in 

15 conjimction with a different postal meter), the discrepancy in the association between 
the envelope watermark and the postal franking mark can be detected by the postal 
authorities, and suitable action taken (e.g., alerting the proper owner of the envelope 
stock of such use). 

The association between the envelope watermark and the postal indicia can be 
20 self-contained (e.g., the association can be demonstrated without reference to external 
resources), or a remote resource can be employed (e.g., a database can specify that 
envelope stock X should only be encountered with digital postage from account Y). 

In a variant embodiment, a franking station can check the watermark already 
existing on an envelope prior to applying postage. If a correct watermark is not 
25 detected, the franking station can decline to apply postage absent supervisory clearance. 
Unauthorized use of corporate mail accounts for use on personal correspondence may 
thereby be curbed. 

In accordance with a fifth aspect of the invention, watermarking on an envelope 
is employed as an element of a franking mark. Such marking can be confined to the 
30 comer location typically associated with postage, but need not be so limited. If the 

marking extends across the entire envelope - on one side or both - machine processing 
of the mail by the postal system can be facilitated by obviating the need for positioning 
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the envelope in a certain orientation for reading. In one particular embodiment, an 
embedded calibration signal associated with certain watermarks (c.f., the cited patent 
documents) can be used to orient a digital image of the envelope for both watermark 
reading as well as for otitier machine processing. In other embodiments, printed 
5 features of a franking mark (e.g., vertical and horizontal lines) can be used like 
graticules to aid in establishing the skew of a watermark printed with the franking 
mark, thereby aidmg decoding of the watermark. 

To make clear to office personnel that postage has been applied to such an 
envelope, it is generally desirable that the marking be visible. This can be achieved by 
10 increasmg the ampUtude of the watermark signal so that it appears as a patterned tile 
(or other shape). Or the watermark can be imperceptible, and other indicia added to 
indicate tiiat postage has been applied (e.g., text stating **Posted with $0.33, printed in 
the same area as the watermark, or in a different area). 

In still other arrangements, a conventional 2D barcode franking mark is subtly 
15 changed to, itself, imperceptibly carry the watermark. 

The information encoded in the franking (or other) watermark can represent a 
great variety of data. The amoimt of postage encoded, the date of encoding, the 
sender's name, address and zip code, the recipient's name, address and zip code, etc., 
can all be indicated. 

20 In some embodiments, all such information is directly encoded in the 

watermark. In other embodiments, the watermark encodes an abbreviated data set, e.g., 
including a code number. The code number corresponds to additional information that 
can be found in a database record accessed by the code - either maintained by the user, 
by a central authority (e.g., the postal system), or by some remotely accessible 

25 database. 

In accordance with a sixth aspect of the invention, watermarking on an envelope 
serves as a portal to a corresponding internet site or application (which could be local 
on the user's PC). 

As detailed in the assignee's application 09/571,422, filed May 15, 2000 
30 (WO0070585), a watermarked document can be held up to a web cam, or scanned by a 
scanner, and serve to instantly link a user to an Intemet site, to invoke an appUcation, 
etc. (The present assignee offers such services under the Digimarc MediaBridge ^ 
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name.) An envelope marked in this fashion can allow a user to initiate an essentially 
unlimited range of options. 

Consider an envelope having the sender's contact information (name, address, 
zip code, phone number, fax number, email address, etc.) represented by a watermark 
5 (either literally, or referring to a database record). A recipient of the envelope may 
present same to a web cam associated with a personal computer. The camera decodes 
the watermark, finds it is contact information for a person, and in response 
automatically adds the contact information to a contact organizer (e.g., Microsoft 
Outlook) maintained by the computer. 

1 0 Different watermarks may trigger different reactions. Certain of the payload 

bits in the watermark may indicate the type of data represented, and/or the type of 
reaction that is appropriate. Req)onses may be programmed by the sender, so the 
watermark is the same, but the backend system that is linked to the watermark contains 
the programming for what response to invoke. 

1 5 One type of watermark may indicate that the encoded information is contact 

information that is available for loading iato a recipient's contact organizer, A second 
type of watermark may indicate that a delivery confirmation message is to be 
dispatched to the sender of the envelope. When such an envelope is presented to the 
recipient's web cam, the associated computer automatically composes an email 

20 message confirming delivery of the envelope, and sends it to an address represented in 
the watermark, 

A third type of watermark may direct a web browser associated with the 
recipient's computer to a destination specified by the watermark. The destination web 
address can provide the recipient with additional information related to the mailing, but 

25 updated to the minute. Advertising mailings can thus link to ordering pages, new sale 
promotions, updated backorder status information, etc. Utility bills can link to 
summary account information showing payments received or owing, month-to-date 
charges, etc. The linked web address may present a form soliciting input or response 
firom the envelope recipient, including survey responses, votes, etc. 

30 The linked resource needn't convey just textual or graphical information. 

Entertainment programming can be similarly invoked, e.g., the delivery of previews of 
tonight's cable television shows, popular music recordings for preview or purchase, etc. 



wo 01/86579 



PCTAJSOl/14920 



• -11-' 

A fourth type of watermark may initiate a replenishment of postage in the 
recipient's digital postage account. 

The foregoiiig is just a small sampling of the myriad functions that can be 
invoked - locally in the recipient's computer, or employing remote resources (e.g., 
5 computers accessed over the internet) - in response to presentation of a mailing to a 
webcam or other imaging device. 

Some watermarks may correspond to several alternative actions. In such case, 
the recipient's computer may present a menu from which the recipient can select the 
desired response. Or the response invoked by presenting the envelope to the web cam 
10 may be made dependent on context or environment in which the presentation is made 
(e.g., time of day, type of device to which web cam is connected - fixed or portable 
computer, wired or wireless, etc.) 

In a variant of the foregoing embodiment, an envelope watermark serves to 
convey an identifier that is used to access a database record having information related 
15 to mail processing or dehvery. In one particular appHcation, the index number 

identifies the recipient. Thus, for example, an envelope can be addressed simply by 
watermarking it with a unique recipient designator, e.g., JOHNQPUBLIC843. 
Processing equipment in the postal system can read the watermark, query a database 
with the designator, and determine thereby the recipient's physical address (e.g., street 
20 address). 

Li some such embodiments, the physical address information obtained by this 
database lookup is printed on the envelope by the postal system for the benefit of the 
ultimate postal delivery person. In other embodiments, the postal delivery person is 
equipped with reader devices that make such printing superfluous. 

25 As noted, an important advantage to this arrangement is that Change of Address 

cards would be a thing of the past. If a person moves across country, a single record in 
the database is changed. All mail to that recipient automatically is directed to the new 
physical address. A Ufetime postal addressing system can thereby be realized. 

In accordance with yet another embodiment, digital watermarks on envelopes 

30 can be apphed by postal metering devices, or by using common office printers (e.g., 
laser, ink-jet). In such systems, the watermark embedding functionality may be 
integrated into the printing device (e.g., by firmware executed by a printer 
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microprocessor, or by dedicated hardware), or can be resident as software on an 
associated computer system. The software is desirably secured against tampering using 
various anti-hacking techniques. The production of the digital watermark may not be 
optional (i.e., it may be applied without user control), and the payload can be tailored in 
5 accordance with the amount of postage, device/software/user information, or other 
application information. 

Such a system may also include an application that calibrates printing of the 
watermark to the user's specific printer or software. For example, the apphcation 
(which may be a software program) may print a predetermined pattern (watermark or 

10 otherwise). The resulting printed media can then be scanned using a scaimer (e.g., a 
digital photocopier or other device) whose transfer function is known. (The apphcation 
may have profile data on several common scanning devices that can be selectively 
invoked (e.g., by the user), depending on the particular scanner used.) The scaimed 
image data is then processed by the apphcation to infer the characteristics of the user's 

1 5 printer or software (e.g., its transfer fimction). Once these characteristics are known, 
the watermarking process can pre-compensate for such printer/software characteristics 
so as to produce a watermark whose attributes are largely independent of the 
printer/software firom which it was generated. (E.g., if a printer exhibits attenuated 
reproduction of high frequency image data, the high frequency components can be pre- 

20 emphasized prior to sending the watermark data to the printer. Similarly, if the dot 
pitch produced by the printer emphasizes particular spatial frequencies, the watermark 
image data can be pre-compensated to de-emphasize such spatial frequencies.) 

In other embodiments, the transfer fimctions of printing systems commonly 
used by users can be pre-characterized by the manufacturer, and appropriate 

25 compensation of watermark printing can be based thereon. Thus, for example, if a user 
is printing postage using a Hewlett-Packard LaserJet 8000DN printer, a first set of pre- 
stored pre-compensation information is utilized. If the user is printing using a Hewlett- 
Packard DeskJet 860C, a second set of pre-compensation information is utiUzed, etc. 
(The specification of the particular printer being used can be left to the user, or it can be 

30 determmed by reference to data available in the computer system (e.g., by reference to 
the printer driver file being employed.)) 
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The net result, again, is to make the printed end-product substantially uniform 
regardless of the idiosyncrasies of particular printing systems. (Further infomiation on 
characterizing the transfer function of devices to assure reUable watermark 
communication is found in copending appUcation 60/173,880, filed December 30, 
5 1999.) 

Li some embodiments, a hardware security device that is used, e.g., to store 
postage value (e.g., a digital vault) may also be employed by the watermark embedding 
system, e.g., as a source of secure and potentially unique data used in encoding the 
digital watermark (e.g., crypto keys, pseudo-random noise data, etc.). In some such 

10 arrangements, the watermark embedding system may make use of data stored in such 
device primarily for another purpose (e.g., a user ID), and can employ such data in 
conjunction with the watermarking operation (e.g., as a seed to a random number 
generator that produces a noise pattern utilized in tiie watermark encoding). 

In accordance with still further embodiments of the invention, the principles 

1 5 described herein are also applicable to other printed value documents (e.g., tickets and 
coupons), especially those that are printed on demand. Such documents may be printed 
at home, at special kiosks (e.g., in-store), or by commercial printing establishments 
(i.e., mass produced). Among other functionality, the watermarks in such value 
documents can be used by investigators to distmguish originals from reproductions (by 

20 use of fragile watermarks), to authenticate documents (e.g., in ticket reading machines), 
and to link to associated internet resources. (An example of the latter is a ticket to a 
sporting or theatrical event that, when presented to a web cam,, allows the user to see an 
actual or virtual view of the sports arena/stage from the perspective of the ticketed 
seat.) 

25 The watermarks on printed value documents can also be Unked to other 

information (e.g., event date, seat number, product code, etc.) that is textually printed 
on the document, or present on the document in some machine readable form (e.g., 
barcodes). In such case, the watermark can be used to detect document alteration by 
checking for discrepancy between the watermark-encoded iiiformation, and that 

30 otherwise conveyed by the document. 

In accordance with still other embodiments, blank paper stock can be digitally 
watermarked so that printed documents formed by later printing on the stock exhibits 



wo 01/86579 



PCT/USOl/14920 



. -14-- 

desired functionality. (The watermark in the blank stock persists through, and is 
detectable notwithstanding, subsequent printing.) 

For example, blank paper stock can be digitally watermarked with a frail 
watermark that permits the original document to be distinguished from photocopies or 
5 other reproductions. Or blank stock used as corporate stationary can be watermarked 
with data serving as an internet link to the corporation's web site. Or serialized sheets 
can be employed by a corporation for sensitive memoranda, allowing a printed 
document to be distinguished from seemingly-identical documents, e.g., permitting the 
document to be traced back to its original intended recipient 
10 As in the examples earher given, the watermark can be formed by ink (e.g., 

speckles, or tinting) or by texture. The watermark can be formed on the paper in bulk - 
in the paper-making process, or can be applied on a per-sheet basis. In the former case, 
the same watermark payload is encoded on large lots of paper, whereas in the second 
case, different watermark payloads can be appUed to different sheets (e.g., serialized 
1 5 paper). (The later process can be performed by high-speed printing machines 
specialized for this purpose, e.g., employing page-width ink-jet arrays.) 

(Watermarking of blank paper stock is referenced in various of the assignee's 
appUcations, including 09/127,502 and 09/619,264, as well as in patent 5,822,436.) 
It will be recognized fliat the arrangements described above can be combined 
20 and hybridized in various ways to economically effect multiple functionality, 
To provide a comprehensive disclosure without unduly lengthening this 
specification, the patents and applications cited herein are incorporated herein by 
reference. 

Having described and illustrated the principles of the invention with reference 
25 to illustrative embodiments, it should be recognized that the invention is not so limited. 

For example, while digital watermarking typically does not leave any human- 
apparent evidence of alteration or data representation, certain of the foregoing 
appUcations do not require this. The markings used may be visible, and even 
conspicuous, without impairing essential ftinctionaUty. Thus, bar codes, data glyphs, 
30 OCR markings, and other machine-readable indicia may be substituted, depending on 
the particular application requirements. 
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While the detailed embodiments were described with reference to desktop 
computers, it is recognized that such devices will increasingly be supplanted by other 
digital appliances, including general purpose personal digital assistants, multifunction 
cell phones, and specialized devices - many of which include integrated optical sensors 
5 (e.g., CCD or CMOS cameras). Moreover, the power and utility of the above-detailed 
embodiments and devices can be further enhanced by employing various wireless 
communications technologies, including the Bluetooth standard. 

The implementation of the watermark encoding and decoding systems is 
straightforward to artisans in the field, and thus not belabored here. Conventionally, 

10 such technology is implemented by suitable software, stored in long term memory (e.g., 
disk, ROM, etc.), and transferred to temporary memory (e.g., RAM) for execution on 
an associated CPU. Li other implementations, the functionality can be achieved by 
dedicated hardware, or by a combination of hardware and software. Reprogrammable 
logic, including FPGAs, can advantageously be employed in certain implementations. 

15 While the specification makes reference to "paper" and "envelopes," these 

terms are used in shorthand fashion to refer to articles delivered by the postal service. 
Thus, postcards (e.g., direct mail cards) and Tyvek articles are meant to be 
encompassed by such references. Postcards may include multiple watermarks, e.g., a 
postal-related mark on the "address side," and an internet-linking mark on the other. 

20 The two marks may be associated or linked in various manners. 

Although not described in the context of existing postal meters, it should be 
recognized that the above-detailed technology is well-sxiited for implementation with 
such devices, as they generally use printing techniques that are suitable for digital 
watermark printing. By retrofitting existing postal meters, a great variety of security 

25 and marketing improvements can readily be provided. 

The reader will recognize that a variety of additional security techniques can be 
employed in conjunction with the arrangements detailed above. For example, in some 
applications, it is useful to encrypt the message encoded in the watermark. Encryption 
provides an additional layer of security to prevent unwanted uses of the encoded 

30 information. Some examples of appUcable cryptographic methods include RS A, DES, 
IDEA (Intemational Data Encryption Algorithm), skipjack, discrete log systems (e.g.. 
El Gamal Cipher), elliptic curve systems, cellular automata, etc. 
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These and other cryptographic methods can be used to create a digital signature 
to place in a watermark message. Public key cryptographic methods employ a private 
and public key. The private key is kept secret, and the pubUc key is distributed. To 
digitally sign a message, the originator of the message encrypts the message with his 

5 private key. The private key is uniquely associated with the originator. Those users 
having a public key verify that the message has originated from the holder of the 
private key by using the pubUc key to decrypt the message. 

The message may be both encrypted and digitally signed using two stages of 
encryption. At the encoder, a digital signature stage encrypts at least part of the 

10 message with a private key. An encryption stage then encrypts the message with a 
public key. The decoder reverses the process. First, a decryption stage decrypts the 
message with a private key corresponding to pubhc key used in the encryption stage at 
the encoder. Then, a second stage decrypts the output of the previous stage with the 
public key corresponding to the private key used to create the digital signature. 

15 Time and date stamping can be used in conjunction with encryption, or 

otherwise (e.g., in a watermark). Metadata can similarly be conveyed. 

If desired, a watermark can be used to track mail (e.g., an envelope or parcel) 
through the deUvery process. At various check points, a camera- or sensor-equipped 
device reads the watermark, extracts an identifier and logs the identifier along with 

20 additional information, such as location, time, etc. This information may be sent and 
maintained in a database that can be queried to determine the deUvery status of the 
mail. Wireless devices can be employed to read watermarks and report status to a 
centralized or distributed database. 

It should be recognized that the particular combinations of elements and 

25 features in the above-detailed embodiments are exemplary only; the interchanging and 
substitution of these teachings with other teachings in this and the incoiporated-by- 
reference patents/appUcations are also contemplated. 

In view of the wide variety of embodiments to which the principles and features 
discussed above can be apphed, it should be apparent that the detailed embodiments are 

30 illustrative only and should not be taken as limiting the scope of the invention. Rather, 
we claim as our invention all such modifications as may come within the scope and 
spirit of the following claims and equivalents thereof. 
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WE CLAIM: 



1 . An original envelope having encoded thereon a fragile digital watermark 
representing plural bits of digital data, said watermark permitting a photocopy thereof 

5 to be distinguished from the origmal. 

2. The envelope of claim 1 in which the watermark is formed with ink. 



3. The envelope of claim 1 in which the watermark is formed by texturing of 
10 the original envelope medium 

4. The envelope of claim 1 that additionally has encoded thereon a second 
digital watermark that withstands at least certain photocopying operations, 

IS 5. The envelope of claim 4 in which the second digital watermark encodes data 

useful for linking to an internet computer site. 

6. The envelope of claim 4 in which flie second digital watermark encodes data 
representing a device or user that produced the document. 

7. The envelope of claim 4 in which one of said watermarks indicates to 
compliant equipment that the envelope should not be reproduced. 

8. The envelope of claim 4 in which the second digital watermark is printed on 
the envelope at the same time as a franking mark. 

9. The envelope of claim 8 in which the second digital watermark is printed on 
the envelope by the same printing assembly used to print said franking mark. 

30 10. The envelope of claim 4 in which at least one of said digital watermarks 

occupies a region that is also occupied by a franking mark printed on said envelope. 
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11. The envelope of claim 4 in which the second watermark is formed on a 
second side of the envelope, opposite a side on which the first watermark is formed. 

12. The envelope of claim 1 in which said digital watermark is printed on the 
5 envelope at the same time as a franking mark, 

13. The envelope of claim 1 in which said digital watermark is printed on the 
envelope by the same printing assembly used to print said fi-anking mark. 

10 14. A blank original substrate suitable for later use in a printing operation to 

produce a printed document, the blank original substrate having encoded thereon a 
fragile digital watermark representing plural bits of digital data, said watermark 
permitting a photocopy thereof to be distinguished firom liie original. 

15 15. The substrate of claim 14 in which the watermark is formed with ink. 

16. The substrate of claim 14 in which the watermark is formed by texturing of 
the substrate mediimi 

20 17. A printed document comprising the substrate of claun 14 that additionally 

has encoded thereon a second digital wateraiark that withstands at least certain 
photocopying operations. 

18. The document of claim 17 in which the second digital watermark encodes 
25 data useful for linking to an internet computer site. 

19. The document of claim 17 in which the second digital watermark encodes 
data representing a device or user that produced the document. 

30 20. The document of claim 17 in which the second watermark is formed on a 

second side of the envelope, opposite a side on which the first watermark is formed. 
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21. The substrate of claim 14 in which the watermark indicates to compliant 
equipment that the envelope should not be reproduced. 

22. An envelope having encoded thereon a fragile digital watermark 
representing plural bits of digital data, said watermark being designed to evidence 
reproduction by scanning and printing. 

23. A method of routing an article of paper mail by a postal authority, 
comprising: 

at said postal authority, reading from said article a machine-readable indicia and 
producing data corresponding thereto; 

consulting a database record to identify a physical address corresponding to at 
least a portion of said data; and 

directing said mail to said physical address. 

24 The method of claim 23 that additionally includes printing said physical 
address on said mail after consulting said database. 

25. The method of claim 23 in which the reading comprises reading with a 
20 sensor comprising rows and colimms of sensing elements. 

26. An envelope having machine readable indicia thereon corresponding to a 
dehvery address, the envelope not having printed thereon a textual delivery address. 

25 27. An envelope having steganographically encoded thereon an optically 

discemable indicia that indicates, to a suitably authorized detector, data that identifies a 
device used in franking said envelope. 



10 



28 An envelope having encoded thereon a fragile digital watermark 
30 representing plural bit of digital data, said watermark not withstanding a photocopy 
operation. 
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29. An envelope having encoded thereon a machine readable indicia that 
indicates, to suitable equipped devices, that image data corresponding to said envelope 
should not be reproduced. 

5 30. An envelope having steganographically encoded thereon a digital 

watermark representing postage. 

3 1 , The envelope of claim 30 in which the steganographic encoding takes the 
form of printing on said envelope, 

10 

32. The envelope of claim 3 1 in which said printing is with an ink designed for 
sensing in the ultraviolet or infrared spectra. 

33.. The envelope of claim 30 in which said encoding takes the form of 
1 5 texturing on the surface of said envelope. 

34. A method comprising: 

receiving an envelope of paper mail through the postal system by an addressee 
at a delivery address specified on the article; 
20 after receipt, presenting the envelope to a processing device that includes an 

optical sensing system; and 

thereby triggering the device to produce a response correspondiug to machine 
readable indicia formed on said envelope. 

25 35. The method of claim 34 in which the response comprises updating a contact 

database on said device with information corresponding to a sender of said envelope. 

36. The method of claim 34 in which the response comprises sending a delivery 
confirmation to a sender of said envelope. 

30 

37. The method of claim 34 in which the response includes directing an internet 
web browser to an internet destination. 
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38. The method of claim 34 that further includes: 

transmitting data produced by said device in response to optical sensing of said 
envelope to a first intemet resource; 
5 receiving additional information or entertainment firom a second internet 

resource; and 

presenting said additional information or entertainment to said addressee. 



39. The method of claim 38 in which said fnst and second intemet resource are 
10 the same intemet address. 

40. The method of claim 38 in which the first and second intemet resources are 
different intemet resources, the first intemet resource responding with an address of 
said second intemet resource. 

41. An envelope having formed thereon two machine readable indicia, a first of 
said indicia being a firanking mark applied by a first party, the second of said indicia 
conveying data associated with an authorized user of said envelope, the first and second 
indicia cooperating to confirm that use of said envelope by the first party is authorized. 



20 



